Fuzzy Systems Are Universal Approximators for Random Dependencies: A Simplified Proof

In many real-life situations, we do not know the actual dependence y = f(x1, . . . , xn) between the physical quantities xi and y, we only know expert rules describing this dependence. These rules are often described by using imprecise (“fuzzy”) words from natural language. Fuzzy techniques have been invented with the purpose to translate these rules into a precise dependence y = f̃(x1, . . . , xn). For deterministic dependencies y = f(x1, . . . , xn), there are universal approximation results according to which for each continuous function on a bounded domain and for every ε > 0, there exist fuzzy rules for which the resulting approximate dependence f̃(x1, . . . , xn) is ε-close to the original function f(x1, . . . , xn). In practice, many dependencies are random, in the sense that for each combination of the values x1, . . . , xn, we may get different values y with different probabilities. It has been proven that fuzzy systems are universal approximators for such random dependencies as well. However, the existing proofs are very complicated and not intuitive. In this paper, we provide a simplified proof of this universal approximation property. 1 Formulation of the Problem It is important to determine dependencies. One of the main objectives of science is to find the state of the world and to predict the future state of the world – both in situations when we do not interfere and when we perform a certain action. The state of the world is usually characterized by the values of appropriate physical quantities. For example: – we would like to know the distance y to a distant star, – we would like to predict tomorrow’s temperature y at a given location, etc. In some cases, we can directly measure the current value of the quantity y of interest. However, in many practical cases, such a direct measurement is not possible – e.g.: – while it is possible to measure a distance to a nearby town by just driving there, 2 M. Afravi and V. Kreinovich – it is not yet possible to directly travel to a faraway star. And it is definitely not possible to measure tomorrow’s temperature y today. In such situations, since we cannot directly measure the value of the desired quantity y, a natural idea is: – to measure related easier-to-measure quantities x1, . . . , xn, and then – to use the known dependence y = f(x1, . . . , xn) between these quantities to estimate y. For example, to predict tomorrow’s temperature at a given location, we can: – measure today’s values of temperature, wind velocity, humidity, etc. in nearby locations, and then – use the known equations of atmospheric physics to predict tomorrow’s temperature y. In some cases we know the exact form of the dependence y = f(x1, . . . , xn), but in many other practical situations, we do not have this information. Instead, we have to rely on experts who often formulate their rules in terms of imprecise (“fuzzy”) words from natural language. Imprecise (“fuzzy”) rules and how they can be transformed into formulas. What kind of imprecise rules can we have? In some cases, the experts formulating the rule are imprecise both about xi and about y. In such situations, we may have rules like this: “if today’s temperature is very low and the Northern wind is strong, the temperature will remain very low tomorrow.” In this case, x1 is temperature today, x2 is the speed of the Northern wind, y is tomorrow’s temperature, and the properties “very low” and “strong” are imprecise. In general, we have rules of the type “if x1 is Ak1, . . . , and xn is Akn, then y is Ak”, where Aki and Ak are imprecise properties. It is worth mentioning that in some cases, the information about xi is imprecise, but the conclusion about y is described by a precise expression. For example, in non-linear mechanics, we can say that when the stress x1 is small, the strain y is determined by a linear formula y = k · x1, with known k, but when the stress is high, we need to use a nonlinear expression y = k · x1 − a · x2 with known k and a. Here, both expressions are exactly known, but the condition when to apply one or another is described in terms of imprecise words like “small”. To transform such expert rules into a precise expression, Zadeh invented fuzzy logic; see, e.g., [1, 4, 5]. In fuzzy logic, to describe each imprecise property P , we ask the expert to assign, to each possible value x of the corresponding quantity, a degree μP (x) to which the value x satisfies this property – e.g., to what extent the value x is small. We can do this, e.g., by asking the expert to mark, on a scale from 0 to 10 to what extent the given value x is small. If the Fuzzy Systems Are Universal Approximators for Random Dependencies 3 expert marks 7, we take μP (x) = 7/10. The function μP (x) that assigns this degree is known as the membership function corresponding to the property P . For given inputs x1, . . . , xn, a value y is possible if it fits within one of the rules, i.e., if: – either the first rule is satisfied, i.e., x1 is A11, . . . , xn is A1n, and y is A1, – or the second rule is satisfied, i.e., x1 is A21, . . . , xn is A2n, and y is A2, etc. Since we assumed that we know the membership functions μki(xi) and μk(y) corresponding to the properties Aki and Ak, we can thus find the degrees μki(xi) and μk(y) to which each corresponding property is satisfied. To estimate the degree to which y is possible, we must be able to deal with propositional connectives “or” and “and”, i.e., to come up with a way to estimate our degrees of confidence in statements A ∨ B and A&B based on the known degrees of confidence a and b of the elementary statements A and B. In fuzzy logic, such estimation algorithms are known as t-conorms (“or”-operations) and t-norms (“and”-operations). We will denote them by f∨(a, b) and f&(a, b). In these terms, the degree μ(y) to which each value y is possible can be estimated as μ(y) = f∨(r1r2, . . .), where rk def = f&(μk1(x1), . . . , μkn(xn), μk(y)). We can then transform these degrees into a numerical estimate y. This can be done, e.g., by minimizing the weighted mean square difference ∫ μ(y)·(y−y) dy, which results in y = ∫ y · μ(y) dy ∫ μ(y) dy . Universal approximation result for deterministic dependencies. For deterministic dependencies y = f(x1, . . . , xn), there are universal approximation results according to which for each continuous function on a bounded domain and for every ε > 0, there exist fuzzy rules for which the resulting approximate dependence f̃(x1, . . . , xn) is ε-close to the original function f(x1, . . . , xn) for all the values xi from the given domain. In practice, we can often only make probabilistic predictions. In practice, many dependencies are random, in the sense that for each combination of the values x1, . . . , xn, we may get different values y with different probabilities. Fuzzy systems are universal approximators for random dependencies as well. It has been proven that fuzzy systems and universal approximators for random dependencies as well; see, e.g., [2, 3]. Remaining problem: can we simplify these proofs. The proofs presented in [2, 3] are very complicated and not intuitive. It is therefore desirable to simplify these proofs. What we do in this paper. In this paper, we provide a simplified proof of the universal approximation property for random dependencies. 4 M. Afravi and V. Kreinovich 2 Towards a Simplified Proof Main idea: how do we simulate random dependencies? To simulate a deterministic dependence y = f(x1, . . . , xn), we design an algorithm that, given the values x1, . . . , xn, computes the corresponding value y. To simulate a random dependence, a computer must also use the results of some random number generators that generate numbers distributed according to some probability distribution. Such generators are usually based on the basic random number generator – which is either supported by the corresponding programming language or even on a hardware level – that generates numbers uniformly distributed on the interval [0, 1]. From this viewpoint, the result of simulating a random dependency has the form y = F (x1, . . . , xn, ω1, . . . , ωm), where F is the corresponding algorithm, xi are inputs, and the values ωj comes from the basic random number generator. In these terns, what does it mean to approximate? In the above terms, to approximate means to find a function F̃ for which, for all possible inputs xi from the given bonded range, and for all possible values ωj , the corresponding value ỹ = F̃ (x1, . . . , xn, ω1, . . . , ωm) are ε-close to the results of applying the algorithm F to the same values xi and ωj . This leads to a simplified proof. The above idea leads to following simplified proof: – due to the universal approximation theorem for deterministic dependencies, for every ε > 0, there exists a system of fuzzy rules for which the value of the corresponding function F̃ is ε-close to the value of the original function F ; – thus, we get a fuzzy system of rules that provides the desired approximation to the original random dependency F .